Effective Support Size

1. Effective size of support Let X be a discrete random variable which can take on values from a finite set X of m elements, with probabilities specified by the probability mass function (pmf) p. The support of X is a set S(p(X)) {p : p i > 0, i = 1, 2,. .. , m}. Let |S(p(X))| denote the size of the support. While pmf p = [0.5, 0.5] makes both outcomes equally likely, the following pmf q = [0.999, 0.001] characterizes a random variable that can take on almost exclusively only one of two values. However, both p and q have the same size of support. This motivates a need for a quantity that could measure size of support of the random variable in a different way, so that the random variable can be according to its pmf placed in the range [1, m]. We will call the new quantity/measure the effective support size (Ess), and denote it by S(p(X)), or S(p), for short. The example makes it obvious that S(·) should be such that S(q) will be close to 1, while to p it should assign value S(p) = 2. Ess should have certain properties, dictated by common sense. P1) S(p) should be continuous, symmetric function. P2) S(δ m) = 1 ≤ S(p m) ≤ S(u m) = m; where u m denotes uniform pmf on m-element support, δ m denotes an m-element pmf with probability concentrated at one point, p m denotes a pmf with |S(p)| = m. P3) S([p m , 0]) = S(p m). P4) S(p(X, Y)) = S(p(X))S(p(Y)), if X and Y are independent random variables. The first two properties are obvious. The third one states that extending support by an impossible outcome should leave Ess unchanged. Only the fourth property needs, perhaps, some little discussion. Or, better, an example. It is reasonable to require the product relationship to hold for independent random variables with arbitrary distributions. The properties P1)-P4) are satisfied by S(p, α) = (m i=1 p α i) 1 1−α , where α is positive real number, different than 1. Note that S(·) of this form is exp 1